Papers with human language processing
Pragmatic inference of scalar implicature by LLMs (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing Large Language Models (LLMs) engage in pragmatic inference of scalar implicature, such as some. |
| Approach: | They investigate how Large Language Models (LLMs) engage in pragmatic inference of scalar implicature, such as some. |
| Outcome: | The proposed models interpret some as pragmatic implicature not all in the absence of context, aligning with human language processing. |
ImaginE: An Imagination-Based Automatic Evaluation Metric for Natural Language Generation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing evaluation methods for natural language generation rely on token-level or embedding-level comparisons with text references. |
| Approach: | They propose to use text-to-image generator to generate an image as the embodied imagination for the text snippet and compute the imagination similarity using contextual embeddings. |
| Outcome: | The proposed metric improves existing evaluation metrics’ correlations with human similarity judgments in both reference-based and reference-free scenarios. |
The Natural Stories Corpus (L18-1)
Copied to clipboard
Richard Futrell, Edward Gibson, Harry J. Tily, Idan Blank, Anastasia Vishnevetsky, Steven Piantadosi, Evelina Fedorenko
| Challenge: | Existing corpora of naturalistic text do not contain the low-frequency syntactic constructions needed to distinguish between theories. |
| Approach: | They propose to compare models of language processing by comparing their ability to predict behavioral and neural measures of processing difficulty to corpora of naturalistic text. |
| Outcome: | The proposed corpus contains low-frequency syntactic constructions while sounding fluent to native speakers. |
Relative Importance in Sentence Processing (2021.acl-short)
Copied to clipboard
| Challenge: | In natural language processing, the relative importance of words is usually interpreted with respect to a specific task. |
| Approach: | They compare the relative importance of words in English language processing by humans and neural language models by using saliency methods. |
| Outcome: | The proposed method could be used to interpret neural language models. |
Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times? (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing studies have shown that larger pre-trained language models with more parameters and lower perplexity are less predictive of human reading times. |
| Approach: | They propose to use a transformer-based model with more parameters and lower perplexity to investigate why these models are less predictive of human reading times. |
| Outcome: | The results show that the larger models with more parameters and lower perplexity are less predictive of human reading times and eye-gaze durations collected during naturalistic reading. |
Fine-Tuning Pre-Trained Language Models with Gaze Supervision (2024.acl-short)
Copied to clipboard
| Challenge: | Existing pre-trained language models lack a gaze module to exploit cognitive signals. |
| Approach: | They propose to integrate a gaze module into pre-trained language models at the fine-tuning stage to exploit cognitive signals. |
| Outcome: | The proposed model improves performance on the GLUE benchmark and standard fine-tuning and text augmentation baselines. |
Design of BCCWJ-EEG: Balanced Corpus with Human Electroencephalography (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent research has focused on the fusion of NLP and neuroscience of language. |
| Approach: | They propose to use a balanced corpus of written Japanese (BCCWJ) annotated with human electroencephalography to improve annotations and annotations. |
| Outcome: | The proposed language resource is annotated with human electroencephalography (EEG) and can improve on annotations, genres, languages, etc. |
Language Models Largely Exhibit Human-like Constituent Ordering Preferences (2025.naacl-long)
Copied to clipboard
| Challenge: | English sentences are typically inflexible vis-à-vis word order, but constituents show far more variability in ordering. |
| Approach: | They compare LLMs with four types of constituent movement to evaluate their performance on heavy NP shift, particle movement, dative alternation, and multiple PPs. |
| Outcome: | The proposed model performs well on four types of constituent movement: heavy NP shift, particle movement, dative alternation, and multiple PPs. |
A Study on How Attention Scores in the BERT Model Are Aware of Lexical Categories in Syntactic and Semantic Tasks on the GLUE Benchmark (2024.lrec-main)
Copied to clipboard
| Challenge: | In the realm of sentence comprehension, human attention is not evenly distributed across all words, indicating systematic variations in language processing. |
| Approach: | They propose to categorize tokens according to their lexical categories and focus on changes in attention scores among these categories during the fine-tuning process for downstream tasks. |
| Outcome: | The proposed model is based on a GLUE benchmark dataset and demonstrates that it assigns more bias to specific lexical categories irrespective of the task. |
Who Relies More on World Knowledge and Bias for Syntactic Ambiguity Resolution: Humans or LLMs? (2025.naacl-long)
Copied to clipboard
| Challenge: | Among various types of ambiguity, this study focuses on syntactic ambiguities, specifically relative 1 Dataset available at https://github.com/PortNLP/ MultiWHO. |
| Approach: | They propose to use a dataset to fine-grained evaluate relative clause attachment preferences in ambiguous and unambiguous contexts. |
| Outcome: | The proposed dataset shows that large language models perform well in unambiguous cases, but lack flexibility in human language processing. |
The Copenhagen Corpus of Eye Tracking Recordings from Natural Reading of Danish Texts (2022.lrec-1)
Copied to clipboard
| Challenge: | Corpora of eye movements during reading of contextualized running text is a way of making such records available for natural language processing. |
| Approach: | They present CopCo, the first eye tracking corpus of its kind for the Danish language. |
| Outcome: | The Copenhagen corpus of eye tracking recordings from natural reading of Danish texts is the first of its kind for the Danish language. |
Do Large Language Models Mirror Cognitive Language Processing? (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models have demonstrated remarkable abilities in text comprehension and logical reasoning. |
| Approach: | They employ Representational Similarity Analysis to measure alignment between 23 LLMs and fMRI signals of the brain. |
| Outcome: | The results show that training strategies affect the LLM-brain alignment. |
CDRNN: Discovering Complex Dynamics in Human Language Processing (2021.acl-long)
Copied to clipboard
| Challenge: | Behavioral and fMRI experiments reveal detailed and plausible estimates of human language processing dynamics . central questions in psycholinguistics concern the mental processes involved in incremental human language understanding . |
| Approach: | They propose a continuous-time deconvolutional regressive neural network that captures time-varying, non-linear, and delayed influences of predictors on the response. |
| Outcome: | The proposed neural network captures time-varying, non-linear, and delayed influences on the response . Behavioral and fMRI experiments show it generalizes better than baselines . |
Long-Range Language Modeling with Selective Cache (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that use transformers to model language cost quadratically increase with sequence length. |
| Approach: | They propose a selective cache which stores key-value pairs from previous contexts. |
| Outcome: | The proposed selective cache outperforms XL cache and compressive cache by considerable margins. |
Linear Recency Bias During Training Improves Transformers’ Fit to Reading Times (2025.coling-main)
Copied to clipboard
| Challenge: | Recent research has shown a strong fit between surprisal values from Transformers and reading times. |
| Approach: | They evaluate a Transformer model that uses a recency bias added to attention scores to improve the fit to human reading times. |
| Outcome: | The proposed model improves on a Transformer that includes a recency bias added to attention scores. |
Robust Open-Vocabulary Translation from Visual Text Representations (2021.emnlp-main)
Copied to clipboard
| Challenge: | MT models have discrete vocabularies and often use subword segmentation to achieve an ‘open vocabulary’. |
| Approach: | They propose to use visual text representations to create continuous vocabularies by processing visually rendered text with sliding windows. |
| Outcome: | The proposed models achieve 25.9 BLEU on character permuted German–English task, compared with traditional models on smaller and larger datasets. |
Causal interventions expose implicit situation models for commonsense language understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Classical psycholinguistic accounts have suggested that world knowledge enters into language understanding through structured schemas called situation models. |
| Approach: | They apply causal intervention techniques to transformer models to analyze performance on the Winograd Schema Challenge . |
| Outcome: | The proposed model performs well on the Winograd Schema Challenge . |
From Human Reading to NLM Understanding: Evaluating the Role of Eye-Tracking Data in Encoder-Based Models (2025.acl-long)
Copied to clipboard
| Challenge: | integrating eye-tracking features into Neural Language Models does not degrade downstream task performance, enhances alignment between model attention and human attention patterns, and compresses the embedding space. |
| Approach: | They used eye-gaze data from the Ghent Eye-Tracking Corpus to investigate how integrating knowledge of human reading behavior impacts Neural Language Models. |
| Outcome: | The proposed approach does not degrade downstream task performance, enhances alignment between model attention and human attention patterns, and compresses the embedding space. |
ScanDL: A Diffusion Model for Generating Synthetic Scanpaths on Texts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Eye movements in reading are a key part of psycholinguistic research, but the lack of eye movement data and its unavailability at application time pose a major challenge for this line of research. |
| Approach: | They propose a novel sequence-to-sequence diffusion model that generates synthetic scanpaths on texts by leveraging pre-trained word representations and jointly embedding both the stimulus text and the fixation sequence. |
| Outcome: | The proposed model outperforms state-of-the-art models in psycholinguistic analysis and is able to exhibit human-like reading behavior. |
Reading Does Not Equal Reading: Comparing, Simulating and Exploiting Reading Behavior across Populations (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing corpora of eye-tracking-while-reading corporata lack diversity, limiting their ability to include primarily native speakers. |
| Approach: | They expand the eye-tracking-while-reading dataset CopCo by incorporating a new dataset of L2 readers with diverse L1 backgrounds. |
| Outcome: | The extended CopCo corpus comprises neurotypical L1 and L1 readers with dyslexia as well as L2 readers reading the same materials. |
Towards Understanding the Relationship between In-context Learning and Compositional Generalization (2024.lrec-main)
Copied to clipboard
| Challenge: | In-context learning is an inductive bias for compositional generalization, but many deep neural architectures struggle with this ability. |
| Approach: | They propose to force a causal Transformer to in-context learn to promote compositional generalization by using earlier examples to generalize to later ones. |
| Outcome: | The proposed model can solve 'ordinary' learning problems by utilizing earlier examples to generalize to later ones, i.e., in-context learning. |
Memory efficiency and resource-rational encoding in sentence processing (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that language models need to be constrained in their use of working memory for context, the analogue to human working memory (WM). |
| Approach: | They propose to inject noise into hidden representations of Transformer-based LMs to capture constraint on memory encoding. |
| Outcome: | The proposed model improves alignment with human reading times and makes them more compressed and categorical. |
On the Effect of Hyperparameters in Language Modeling for Computational Linguistics (2026.acl-long)
Copied to clipboard
| Challenge: | Training language models and examining their linguistic behaviors is a common protocol in computational linguistics for studying linguistic phenomena and modeling human language processing. |
| Approach: | They replicate three prior studies with hyperparameters varied within a practical range and show that modest hyperparametric changes can alter qualitative conclusions about models’ linguistic abilities. |
| Outcome: | The results show that hyperparameter changes can alter qualitative conclusions and reverse the ranking of models. |